Mobile device mode of operation for visually impaired users

ABSTRACT

A mobile device operates in a standard mode of operation, or operates in a visually impaired (VI) mode of operation designed for visually impaired users and in which a display screen of the mobile device is turned off by default. The display screen may thus remain turned off in the VI mode even when conditions under which the display screen is turned on in the standard mode are satisfied. The display screen may further remain turned off in the VI mode even when conditions under which a touch-screen input interface of the mobile device is turned on in either of the standard and VI modes. The mobile device may be configured to automatically transition from the VI mode to the standard mode in response to detecting a gaze directed towards the display screen.

BACKGROUND

Mobile devices such as smart-phones have increasingly large displayscreens. The display screen consumes a large amount of power, andoperation of the display screen causes the battery of the mobile deviceto quickly become depleted. For example, a standard display panel for amobile device can consume around 1 Watt of power when active with normalbrightness. In order to reduce battery usage, the display screen isgenerally automatically dimmed and/or turned off after a short period ofinactivity.

The user interfaces of mobile devices are heavily dependent on thedisplay of information to users. Indeed, the user interfaces generallyrely on touch-screen technology for receiving user input. In order touse touch-screen input, the display screen must be powered on to displaycontent and virtual buttons for selection by the user, and to detect theuser's touch on the screen. The display must thus generally be poweredon at any time that information is displayed to the user, and any timethat user input is expected from the user.

Visually impaired users of mobile devices, such as blind users, use thetouch-screen interface of their mobile devices to interact with themobile devices. Visually impaired users, however, may not generallyrequire the display screen of their mobile devices to be powered on asthey interact with the touch-sensitive input interface. A need thereforeexists for enabling visually impaired users of mobile devices to poweroff the display screens of their mobile devices to as to reduce powerconsumption. The turning off of the display screen can additionallyavoid other persons from viewing information normally displayed on thescreen.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations in accord withthe present teachings, by way of example only, not by way of limitation.In the figures, like reference numerals refer to the same or similarelements.

FIGS. 1A and 1B are state diagrams illustratively showing transitionsbetween the various modes of operation of a mobile device.

FIGS. 2A and 2B are flow diagrams illustratively showing processes fortransitioning between modes of operation in a mobile device.

FIG. 3A is a flow diagrams illustratively showing operations that may beperformed when a mobile device powers up from a powered-off state intothe standard mode or the VI mode of operation.

FIG. 3B is a flow diagram illustratively showing operations that may beperformed when a mobile device operating in the VI mode receives a usercommand to turn on the display screen.

FIGS. 4A-4G are illustrations of mobile devices operating in differentmodes of operation, and receiving user commands for changing the mode ofoperation of the mobile device.

FIG. 5 is a high-level functional block diagram of an exemplary mobiledevice as may operate in the standard and VI modes of operation.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, the present teachings may be practicedwithout such details. In other instances, well known methods,procedures, components, and/or circuitry have been described at arelatively high-level, without detail, in order to avoid unnecessarilyobscuring aspects of the present teachings.

The various methods and system disclosed herein relate to providing anoperating mode for visually impaired users of mobile devices, such as anoperating mode that keeps the display screen powered off by default inorder to reduce the power consumption of the mobile devices.

Visually impaired persons use mobile devices, such as smart-phones, fora similarly wide range of uses as non visually impaired users. Visuallyimpaired users commonly use mobile devices for placing and receivingphone calls, accessing applications and/or the Internet, and listeningto music, podcasts, movies, or other audio or audio-visual content.Users, whether visually impaired or not, interact with and control themobile devices through user input interfaces that commonly take the formof touch-screen input interfaces. In a touch-screen interface,information is displayed for the user on the display screen, and caninclude one or more buttons for receiving input from the user. Inresponse to the user touching a touch-sensitive sensor overlaid on thedisplay screen (or forming part of the screen), the touch-sensitivesensor detects the location of the user's touch, determines whether thedetected location is associated with any button or other function, andperforms the function associated with the activated button or determinedfunction.

While non visually impaired users rely heavily on visual informationdisplayed on a display screen of the mobile device in interacting withand obtaining information from the mobile device, blind users as well ascertain other visually impaired users do not generally use the displayscreen in their interactions with the mobile device. For such users, thedisplay screen is not needed when they interact with the mobile device,and the display screen thus unnecessarily consumes the limited powerstored in the mobile device battery. In fact, the display screen mayinadvertently enable other persons to view information displayed on thescreen, and the display of content on the screen may thus presentprivacy concerns for visually impaired users.

A mobile device may thus advantageously enable the display screen of themobile device to be turned off by default. The display screen may beturned off by default when the mobile device operates in a visuallyimpaired (VI) mode of operation designed for visually impaired users,while the display screen operates normally when the mobile deviceoperates in a standard mode of operation.

The mobile device is configured such that it can separately turn on thedisplay screen and the touch-sensitive input interface. The mobiledevice can thus simultaneously turn both the display screen and thetouch-sensitive input interface on or off, or the mobile device can turnone on while the other remains turned off. During operation in thestandard mode, both the display screen and the touch-sensitive inputinterface are generally turned on or turned off concurrently. However,during operation in the VI mode, the display screen generally remainsturned off—even when the touch-sensitive input interface is turned on.

A user of the mobile device can select whether the mobile deviceoperates in the standard mode or in the VI mode, and can alternatebetween the modes of operation by selectively activating one mode or theother.

FIG. 1A is a state diagram 100 illustratively showing transitionsbetween the various modes of operation of a mobile device. Inparticular, the state diagram 100 shows two modes of operation of themobile device: the standard mode 101 and the VI mode 103. The standardmode 101 is the default mode of operation of the mobile device, such asthe mode in which the device operates when first turned on (e.g.,following production and/or installation of the operating system).

When operating in the standard mode 101, the display screen of themobile device is powered and turned on under a variety of conditions. Ingeneral, in the standard mode 101, the display screen is turned on todisplay information to the user when a button or key of the mobiledevice is activated. For example, the display screen is turned on in thestandard mode 101 in response to a user pressing a power button toboot-up the device (from being powered off). The display screen isfurther turned on in the standard mode 101 in response to a userpressing any button or key to wake-up the device (from being in an idlestate). The display screen may also be turned on in the standard mode101 in response to the device booting-up or waking-up automatically,such as in response to a scheduled reminder, timer, or other alarm. Ineach of these instances, the device is powered up from a low power state(e.g., an “off” state or an “idle” state, such as a locked, standby, orsleep state). In the low power state, the display screen is turned off.In response to being powered up from the low power state in the standardmode 101, the display screen is turned on. However, the display screenmay turn off after a pre-determined period of inactivity (e.g., a periodof 15 s during which no user input is received by the mobile device, oranother user-selectable length of time) or in response to a user lockcommand, a user idle command (e.g., a sleep or lock command), or a powerdown command.

In general, in the standard mode 101, the touch-sensitive inputinterface is turned on and off at substantially the same times (and/orfor substantially the same periods of time) as the display screen. Thus,the touch-sensitive input interface is generally turned on, and isactive and ready to sense a user's touch command, whenever the displayscreen is turned on. Conversely, the touch-sensitive input interface isgenerally turned off, and cannot sense users' touch, whenever thedisplay screen is turned off.

When the mobile device operates in the VI mode 103, the display screenof the mobile device remains turned off by default, such that it remainsoff (and powered down) at substantially all times. Thus, even when oneof the conditions that trigger the display screen to be turned on in thestandard mode 101 is satisfied, the display screen remains turned off inthe VI mode 103. For example, the display screen is not turned on in theVI mode 103 in response to a user pressing a power button to boot-up thedevice (from being powered off). The display screen is further notturned on in the VI mode 103 in response to a user pressing any buttonor key to wake-up the device (from being in an idle state).

In the VI mode 103, however, the touch-sensitive input interface maynonetheless be turned on or activated under substantially the sameconditions that trigger the display screen (and the touch-sensitiveinput interface) to be turned on in the standard mode 101. As such, inthe VI mode 103, the touch-sensitive input interface is turned on andmonitors for user touch input in response to a user pressing a powerbutton to boot-up the device (from being powered off). Thetouch-sensitive input interface is further turned on in the VI mode 103in response to a user pressing any button or key to wake-up the device(from being in an idle state). The touch-sensitive input interface maybe turned back off after a pre-determined period of inactivity (e.g., aperiod of 15 s during which no user input is received by the mobiledevice) or in response to a user lock command, a user idle command(e.g., a sleep or lock command), or a power down command.

The display screen generally remains turned off in the VI mode 103, anddisplays no information to the user. In particular, in the VI mode 103,the display screen remains turned off even when a user is activelyengaging with the touch-sensitive input interface.

Similarly to the standard mode 101, the mobile device can enter a lowpower state, such as a powered-off state or an idle state (e.g., alocked, standby, or sleep state), when operating in the VI mode 103. Inthe low power state, the display screen remains turned off while thetouch-screen user interface is also turned off. The mobile device mayenter the low power state when operating in the VI mode 103 in much thesame ways as the device enters the low power state when operating in thestandard mode 101. In particular, the device can enter a low power stateafter a pre-determined period of inactivity (e.g., a period of 15 sduring which no user input is received by the mobile device) or inresponse to a user lock command, a user idle command (e.g., a sleep orlock command), or a power down command.

The standard mode 101 and VI mode 103 may further provide users withdifferent user interfaces. In the case of the VI mode 103, for example,the user interface may be adjusted to provide enhanced non-visual formsof feedback to users. Thus, in the VI mode 103, screen reader softwaremay be automatically turned on (or turned on by default) to cause textthat would usually be displayed on the display screen to be read tousers. The screen reader software may additionally provide audiodescriptions of certain elements displayed on a screen, such asdescription of the positions of frames that would be displayed on thedisplay screen if the display screen were turned on, or of virtualbuttons that the user may activate on the touch-sensitive inputinterface (e.g., “OK button in lower left corner of screen; Cancelbutton in lower right corner of screen”). In contrast, screen readersoftware may be turned off by default when operating in the standardmode 101, and/or may not provide audio descriptions of non-text elementsdisplayed on the screen.

In addition to audio feedback, operation in the VI mode 103 may provideenhanced haptic feedback. In one example, haptic feedback can be used toprovide confirmation to a user that the user's selection of a virtualbutton through the touch-sensitive input interface was received, forinstance by issuing a haptic pulse in response to and shortly afterreceiving the user's selection. In another example, haptic feedback canbe used to provide confirmation that a user's finger contacting thetouch-sensitive input interface is located above a frame, button, oricon, for instance by issuing a haptic vibration when the finger islocated above the frame. In contrast, haptic feedback may generally beturned off by default when operating in the standard mode 101.

In the VI mode 103, the user interface of the mobile device may furtherautomatically activate a spoken command input interface to enable themobile device to receive input commands spoken by the user and receivedthrough a microphone. The spoken command input interface may be used toreceive dictation from a user, and/or to receive commands for performingtasks on the mobile device (e.g., to select an icon, open a menu, or thelike). The spoken command input interface may be deactivated by defaultin the standard mode 101.

As shown in state diagram 100, the mobile device can transition betweenthe standard mode 101 and the VI mode 103 of operation in response to anactivation of either mode by a user. Thus, when operating in thestandard mode 101, the user may activate the VI mode to cause atransition into the VI mode 103. Conversely, when operating in the VImode 103, the user may activate the standard mode to cause a transitioninto the standard mode 101.

When the mobile device operates in the VI mode 103, the display screenremains powered down at substantially all times. The display screen maynonetheless be activated at selected times and for limited purposes. Forexample, immediately following a boot-up or wake-up operation, thedisplay screen may be temporarily activated for a limited period of timefor the express purpose of providing a sighted user with a visualnotification indicating that the mobile device is in the VI mode 103(e.g., by displaying an on-screen pop-up notification indicating thatthe mobile device is in the VI mode). The notification can provide theuser with an opportunity to activate a standard mode 101 (e.g., bydisplaying an on-screen prompt for activating the standard mode 101) andmay indicate the amount of time remaining before the mobile deviceautomatically enters VI mode 103 if the standard mode 101 is notactivated by the user. During display of the visual indication and/oron-screen prompt, no other information is displayed on the displayscreen. The display screen may be turned off after a pre-determinedlimited period of time has elapsed.

FIG. 1B is a second state diagram 150 illustratively showing transitionsbetween the various modes of operation of a mobile device in moredetail. In addition to the standard mode 101 and VI mode 103 shown indiagram 100, diagram 150 shows transitions into and out of the idlestate 155 (e.g., a sleep, standby, or locked state) and the powered-offstate 157.

As shown, the mobile device enters the idle state 155 from either thestandard mode 101 or the VI mode 103 in response to a time-out or acommand to set the mobile device to idle. In the idle state 155, boththe display screen and the touch-sensitive input interface are turnedoff. The mobile device remains in the idle state 155 until an activatingevent is detected. An activating event may include a user pressing a keyof the mobile device to cause the mobile device to be woken-up, an alarmbeing activated or a timer expiring, or the like. When the activatingevent is detected, the mobile device wakes-up and automaticallydetermines the previous mode of operation in which the mobile deviceoperated immediately prior to entering the idle state 155. In responseto determining that the previous mode was the standard mode 101, themobile device is woken-up and made to operate in the standard mode 101.In response to determining that the previous mode was the VI mode 103,the mobile device is woken-up and made to operate in the VI mode 103.

The mobile device can alternatively enter the powered-off state 157 fromeither the standard mode 101 or the VI mode 103 in response to a user'spower-down command, a power-down trigger (e.g., a restart trigger), or abattery reaching a critically low level. In the powered-off state 157,the display screen, the touch-screen user interface, as well as allother parts of the mobile device are turned off. The mobile deviceremains in the powered-off state 157 until a power button is activatedor a charging cable is connected. When the power button is activated orthe charging cable is connected, the mobile device powers-up andautomatically determines the previous mode of operation in which themobile device operated immediately prior to powering down into thepowered-off state 157. In response to determining that the previous modewas the standard mode 101, the mobile device is powered-up and made tooperate in the standard mode 101. In response to determining that theprevious mode was the VI mode 103, the mobile device is powered-up andmade to operate in the VI mode 103.

In some examples, the mobile device operating in the VI mode may providenon-visual user interface features to users that are not generallyavailable in the standard mode. For example, when operating in the VImode, the mobile device may provide enhanced haptic feedback in responseto touch input from users, provide alternate control methods based onhand gestures using the touch-sensitive input interface, on free-spacegestures measured using an accelerometer and/or gyroscope mounted in themobile device, and on voice recognition or the like. Operation in the VImode may further cause audio triggering technology, such as a highlyenergy efficient audio-monitoring circuit that enables the mobile deviceto continuously monitor a microphone for voice commands uttered by auser, to be powered such that users can control the mobile device usingoral commands.

FIGS. 2A and 2B are flow diagrams illustratively showing processes fortransitioning between modes of operation in a mobile device. Theprocesses are illustrative and may correspond to processes that arepre-established in the mobile device. However, other processes(including user-defined processes) may alternatively be used totransition between modes of operation. FIG. 2A shows a method 200 fortransitioning into the VI mode 103, while FIG. 2B shows a method 250 fortransitioning into the standard mode 101.

In accordance with method 200, a user may begin a process for activatingthe VI mode 103 using a number of commands. The user may issue an oralcommand for activating the VI mode 103 as shown in operation 201, forexample by saying a pre-determined phrase (e.g., “activate VI mode”)while the mobile device is listening for the pre-determined phrase. Theuser may alternatively issue a pre-determined gesture command throughthe touch-screen user interface of the mobile device as shown inoperation 203, such as a two-finger hold gesture (e.g., by holding twoadjacent fingers stationary on the screen for 3 s) or other multi-fingergesture. The user may alternatively access a setup menu of the mobiledevice in order to select a menu option for changing the operating modeof the mobile device, or use a motion-based command (e.g., by performinga pre-determined hand motion while holding the mobile device, such as bygently shaking the mobile device in a horizontal plane for 2 s) in orderto change the operating mode. In some situations, in response toreceiving the user command for changing the operating mode, the mobiledevice may determine a current mode of operation of the mobile device(operation 205). Based on the determination, the mobile device mayeither proceed with operation 207 of method 200 if the current mode isthe standard mode 101, or with operation 257 of method 250 (shown inFIG. 2B) if the current mode is the VI mode 103.

In operation 207, the mobile device seeks confirmation from the userregarding the request for changing the mode of operation. As part ofseeking user confirmation, the mobile device provides feedback to theuser indicating that the request has been received, and prompting theuser for confirmation. The feedback can take various forms. As shown,one or more of haptic, audio, and visual feedback can be used. Forexample, a short haptic pulse can be issued, an auditory notificationcan be issued (e.g., “entering VI mode, say yes or tap screen toconfirm”), and/or a pop-up notification can be issued (e.g., a promptfor “Entering VI mode, select yes or tap to confirm”). Note that thehaptic and audio feedback enable visually impaired users to be apprisedof impending changes in operating mode.

If the user issues a confirmation command, for example by tapping thetouch-screen user interface with two fingers (operation 209) or saying apre-determined command (operation 211) before a pre-determined timeperiod has lapsed, the mobile device proceeds to issue a finalconfirmatory haptic feedback in operation 213 (e.g., a long hapticpulse) before changing the operation mode of the mobile device to the VImode (operation 215). Alternatively or additionally, a motion-basedcommand can be used to issue the confirmation command (e.g., inoperation 209 or 211), and audio feedback can be used instead of or inaddition to haptic feedback in operation 213 (e.g., in the form of atone descending in frequency). If no confirmation command is receivedwithin the pre-determined time period (operation 217), or if a commandto cancel the change in modes is received from the user, the mobiledevice does not change mode of operation and instead continues tooperate in the standard mode (operation 219).

In accordance with method 250 of FIG. 2B, a user of the mobile devicecan activate a transition back into the standard mode of operation usinga similar process. The user may issue an oral command for activating thestandard mode 101 as shown in operation 251, issue a pre-determinedgesture command (e.g., a two finger hold) as shown in operation 253,access an appropriate setup menu of the mobile device, or issue apre-determined motion command (e.g., shaking the mobile device in ahorizontal plane for 2 s). In some situations, in response to receivingthe user command for changing the operating mode, the mobile devicedetermines a current mode of operation of the mobile device (operation255) so as to determine whether to proceed with operation 257 of method250 (e.g., if the current mode is the VI mode 103) or operation 207 ofmethod 200 (e.g., if the current mode is the standard mode 101).

In operation 257, the mobile device seeks confirmation from the userregarding the request for changing the mode of operation. As part ofseeking user confirmation, the mobile device provides haptic, audio, orvisual feedback to the user so as to prompt the user for confirmation.Operation 257 may include turning on the display screen to provide thevisual feedback. If the user issues a confirmation command, for exampleby tapping the touch-screen user interface with two fingers (operation259) or saying a pre-determined command (operation 261) before apre-determined time period has lapsed, the mobile device proceeds toissue a final confirmatory haptic feedback in operation 263 beforechanging the mode of operation of the mobile device to the standard mode(operation 265). Alternatively or additionally, a motion-based commandcan be used to issue the confirmation command (e.g., in operation 259 or261), and audio feedback can be used instead of or in addition to hapticfeedback in operation 263 (e.g., in the form of a tone ascending infrequency). If no confirmation command is received within thepre-determined time period (operation 267), or if a command to cancelthe change in modes is received from the user, the mobile device doesnot change operation mode and instead continues to operate in the VImode (operation 269).

FIG. 3A is a flow diagram 300 illustratively showing operations that maybe performed when a mobile device powers up from a powered-off stateinto the standard mode 101 or the VI mode 103. In diagram 300, themobile device powers up and determines the previous mode of operation inwhich the mobile device operated immediately prior to powering down(operation 301). In response to determining that the previous mode wasthe standard mode, the mobile device is powered-up and made to operatein the standard mode (operation 303). In response to determining thatthe previous mode was the VI mode, the mobile device is powered-up andthe display screen is turned on. The mobile device displays a temporaryprompt in operation 305 notifying a user that the device is beingpowered-up and made to operate in the VI mode. The notification canprovide the user with an option for causing the device to power-up andbe made to operate in the standard mode instead. If no user input isreceived in response to the prompt, the VI mode is activated inoperation 307 and the display screen is turned off. However, if the userselects the option for powering the device up in the standard mode inoperation 305, the mobile device proceeds to operation 303 and operatesin the standard mode.

In some embodiments, the mobile device may be configured to enable auser to temporarily turn on the display screen when operating in the VImode. In one example, the temporary turning on of the display screen mayenable a visually impaired user to show pictures or other visual contentto a non-visually impaired user (e.g., a friend or family member),and/or to seek assistance in operating the mobile device from anon-visually impaired user (e.g., a friend, or technical supportperson). In general, following the display screen being temporarilyturned on in the VI mode, the display screen remains turned on until auser command to turn off the display screen is received, or until themobile device enters a low power state (e.g., idle or off state),whichever occurs first. In one example, the display screen may remainturned on for a pre-determined or user-selected period (e.g., 3 minutes)following receipt of a command to temporarily turn on the displayscreen, and may automatically be turned off at the end of thepre-determined or user-selected period unless a follow-on user requestto maintain the display screen temporarily turned on is received.

FIG. 3B is a flow diagram 350 illustratively showing operations that maybe performed when a mobile device operating in the VI mode receives auser command to turn on the display screen. In diagram 350, the mobiledevice operating in the VI mode is operative to turn on the displayscreen under various conditions. In one situation, the mobile device canturn on the display screen in response to receiving a request or commandfrom a user to turn on the display screen (operation 351). The requestcan come in the form of an oral command (e.g., “turn display on”), apre-determined gesture command (e.g., a three-finger hold gesture inwhich three adjacent fingers are held stationary on the screen for 3 s),or by accessing a corresponding menu option of the mobile device. Inresponse to the request, the mobile device may prompt the user forconfirmation that the user wants to turn on the display screen(operation 352), for example by issuing one or more of an audio, visual,or haptic notification and waiting for confirmation from the user (e.g.,using operations similar to operations 207-213 of FIG. 2A). If the userconfirms the request, the display screen in temporarily turned on(operation 353) while the device continues to operate in the VI mode.Note that the display screen is only temporarily turned on. As such, assoon as the mobile device enters an idle state or a powered-off state(operation 355), or as soon as the display screen is otherwise caused tobe turned off, the display screen returns to its default operating modeof being turned off while the mobile device operates in the VI mode(operation 357). The user can once again temporarily turn on the displayscreen by repeating operation 351. Additionally, when the display screenis temporarily turned on, the user can turn off the display screen byspeaking an oral command (e.g., “turn display off”), a pre-determinedgesture command (e.g., the three-finger hold gesture), or by accessing acorresponding menu option of the mobile device.

In another situation, the mobile device can turn on the display screenin response to automatically determining a need for the display screen(operation 359), such as by detecting that a gaze is directed towardsthe display screen of the mobile device. In this situation, the mobiledevice may continuously monitor whether a gaze of a user of the mobiledevice is directed towards the display screen of the mobile device whenthe mobile device operates in the VI mode, for example by using aforward facing camera of the mobile device that tracks eyes (and/oririses, pupils, or glints in eyes) of users looking at the displayscreen. When a gaze is detected as being directed towards the displayscreen and is detected to be sustained (e.g., a gaze that is steadilydirected at the display screen for at least a 2 s period or anotherpredetermined length of time), the mobile device may prompt the user forconfirmation that the user wants to turn on the display screen(operation 352). The prompt may be an audio prompt to which the user canrespond through a spoken command or selection of a response option onthe touch-sensitive input interface. The prompt can alternatively be avisual prompt displayed on the display screen after temporarily turningon the display screen, and including response option buttons displayedon the screen for selection using the touch-sensitive user interface. Ifconfirmation is received, the mobile device proceeds with operations353-357 as described above.

In some examples, the mobile device monitors whether a gaze of a user isdirected towards the display screen (or monitors for triggers used inautomatically turning on the display screen) under specifiedcircumstances. For instance, the mobile device may only monitor forusers' gazes when particular applications are currently active on themobile device. An application is currently active on the mobile deviceif the application is running on the device and is controlling the userinterface of the device (e.g., the application is actively running onthe mobile device rather than running in the background, and the mobiledevice is not operating in a low power state). A currently activeapplication can, for example, be an application that a user of themobile device has activated by selecting an icon for the application onthe mobile device's home screen. In one example, the particularapplications may be applications that are inherently associated withvisual content, such that the mobile device monitors for user gazes onlywhen applications that are inherently associated with visual content,such as photo viewing/sharing applications, video players, or the like,are actively being used on the mobile device. Conversely, in theexample, the mobile device may not monitor for user gazes whennon-visual applications, such as an audio-book reader, a music or radioplayer, or the like, are in active use. The particular applications maybe applications included in a pre-determined list of applicationscompatible with the automatic turning on of the display screen when asustained gaze is detected (e.g., such as applications that areinherently associated with visual content in the example above), and/ora list of user-selected applications among the applications that areinstalled on the mobile device.

In another example, the mobile device can turn on the display screen inresponse to automatically determining a need for the display screen whenoperating in the VI mode (operation 359) such as determining that anapplication that is inherently associated with visual content is activeon the mobile device, whether or not a gaze is detected. In the otherexample, the prompt (operation 352) may be provided to the user onlyafter the application has been active for a pre-determined period oftime (e.g., 5 seconds), such that the prompt is not provided when anapplication is erroneously activated and is promptly de-activated by theuser.

Conversely, the mobile device can automatically transition from thestandard mode to the VI mode so as to turn off the display screen inresponse to automatically determining that the display screen may pose adistraction to a user when the mobile device operates in the standardmode. For example, the mobile device may determine that a user isdriving or otherwise engaging in an activity in which visualdistractions should be minimized. The determination can, in one example,be based on determining that an application that is associated withdriving (e.g., an application providing turn-by-turn driving directionsto a user of the mobile device) is active on the mobile device. Thedetermination can further be based on sensor input (e.g., a sensordetermining that the mobile device is docked in a vehicle-dock). Theautomatic transition from the standard mode to the VI mode may activatea screen reader or other audio-based interface in addition to turningoff the display screen.

In an alternate embodiment, in response to detecting that a gaze isdirected towards the display screen of the mobile device (e.g., as inoperation 359), the mobile device can initiate a change in operatingmode of the mobile device. In this situation, the mobile deviceinitiates a change of mode from the VI mode to the standard mode, forexample by prompting the user for confirmation that the user wants tochange the mode of operation (e.g., similarly to operations 257-265 ofFIG. 2B) prior to proceeding with operation in the standard mode uponreceiving confirmation, or reverting to the VI mode if no confirmationis received.

FIGS. 4A-4G are illustrations of mobile devices operating in differentmodes of operation, and receiving user commands for changing the mode ofoperation of the mobile device.

FIG. 4A shows a mobile device operating in the standard mode ofoperation in which the display screen is powered on and displaying ahome screen. While operating in the standard mode, a user touches thetouch-screen user interface with a two-finger hold gesture in order toinitiate the activation of the VI mode (e.g., as in operation 203 ofFIG. 2A). In response to the two-finger hold gesture, the mobile devicedisplays a confirmation screen shown in FIG. 4B prompting the user forconfirmation of the command to activate the VI mode (e.g., as inoperation 207). In response to the user selecting an “Ok” option orsaying the command “Activate”, the mobile device transitions to the VImode (e.g., as in operation 215). Alternatively, the user can select a“Cancel” option or wait a predetermined time-out period to remain in thestandard mode (e.g., as in operation 219).

FIG. 4C shows a mobile device operating in the VI mode of operation inwhich the display screen is powered off but the touch-screen userinterface is nonetheless powered up and active. While operating in theVI mode, the user touches the touch-screen user interface with atwo-finger hold gesture in order to initiate the activation of thestandard mode (e.g., as in operation 253 of FIG. 2B). In response to thetwo-finger hold gesture, the mobile device activates the display screento display the confirmation screen shown in FIG. 4D (e.g., as inoperation 257). The display screen of FIG. 4D prompts the user forconfirmation of the command to activate the standard mode. The user canselect an “Ok” option or say the command “Activate” in order totransition to the standard mode (e.g., as in operation 265) and causethe mobile device to display content on the display screen.Alternatively, the user can select a “Cancel” option or wait apredetermined time-out period to remain in the VI mode (e.g., as inoperation 269) and cause the display screen to be powered back off.

FIG. 4E shows a mobile device being powered up from a powered-off state.The mobile device has determined that it should power-up in the VI mode,for example in response to determining that the mobile device wasoperating in the VI mode immediately before being powered off (e.g., asin operation 301 of FIG. 3A). As part of the powering-up process, themobile device turns on the display screen so as to display a promptthereon (e.g., as in operation 305). As shown in FIG. 4E, the mobiledevice displays a temporary prompt notifying a user (e.g., sighted user)that the device is being powered up in the VI mode. The prompt providesan option for powering the device up in the standard mode instead. If nouser input is received in response to the prompt, the VI mode isactivated (e.g., as in operation 307) and the display screen in turnedoff. However, if user input for activating the standard mode isreceived, the mobile device can activate the standard mode instead ofthe VI mode (e.g., as in operation 303).

FIG. 4F shows a mobile device operating in the VI mode of operation inwhich the display screen is powered off but the touch-screen userinterface is nonetheless powered up and active. While operating in theVI mode, the user touches the touch-screen user interface with athree-finger hold gesture in order to initiate the temporary turning onof the display screen (e.g., as in operations 351-357 of FIG. 3B). Inresponse to the three-finger hold gesture, the mobile device activatesthe display screen to display a confirmation screen (not shown)prompting the user to confirm the request to temporarily turn on thedisplay screen (e.g., as in operation 352).

FIG. 4G illustrates a mobile device operating in the VI mode ofoperation and having detected a gaze directed towards the display screenof the mobile device (e.g., as in operation 359 of FIG. 3B). In responseto monitoring for and detecting the gaze, the mobile device activatesthe display screen to display the confirmation prompt of FIG. 4Gprompting the user to confirm whether the display screen should betemporarily turned on (e.g., as in operation 352).

FIG. 5 provides a block diagram illustration of an exemplary mobiledevice 10. Although the mobile device 10 may be a smart-phone, a tabletcomputer, or may be incorporated into another device such as a personaldigital assistant (PDA) or the like, for discussion purposes theillustration shows the mobile device 10 in the form of a handset. Themobile device 10 is powered by a battery 118, and is operative tocommunicate wirelessly with a network 11 such as a mobile wirelesscommunication network (e.g., a cellular communication network), a Wifinetwork, or the like.

The handset embodiment of the mobile device 10 functions as a normaldigital wireless telephone station. For that function, the device 10includes a microphone 102 for audio signal input and a speaker 104 foraudio signal output. The microphone 102 and speaker 104 connect to voicecoding and decoding circuitry (vocoder) 106.

For digital wireless communications, the device 10 also includes atleast one digital transceiver (XCVR) 108 connected to an antenna 110.Today, the device 10 can be configured for digital wirelesscommunications using one or more of the common network technology types.The transceiver 108 provides two-way wireless communication ofinformation, such as vocoded speech samples and/or digital information,in accordance with the technology of the network 11. The transceiver 108also sends and receives a variety of signaling messages in support ofthe various voice and data services provided via the mobile device 10and the communication network 11. In some examples, in addition to atransceiver 108 configured for communication through a mobile wirelesscommunication network (e.g., a GSM, 3G, 4G, or LTE network, or thelike), an additional transceiver is connected to a separate antenna toprovide communication capabilities through a different type of networksuch as a Wifi network.

The mobile device 10 includes a display 122, which the microprocessor112 controls via a display driver 124, to present visible outputs to thedevice user. The device 10 also includes a touch/position sensor 126comprising the touch-sensitive input interface. The touch-sensitivesensor 126 is relatively transparent, so that the user may view theinformation presented on the display 122 through the sensor 126. A sensecontroller circuit 128 senses signals from elements of thetouch/position sensor 126 and detects occurrence and position of eachtouch of the screen formed by the display 122 and sensor 126. The sensecircuit 128 provides touch position information to the microprocessor112, which can correlate that information to the information currentlydisplayed via the display 122, to determine the nature of user input viathe screen.

The display 122 and touch sensor 126 (and possibly a limited number ofkeys 130, if included) are the physical elements providing the textualand graphical user interface for the mobile device 10. The microphone102 and speaker 104 may be used as additional user interface elements,for audio input and output.

The microprocessor 112 is operative to selectively and separately turnon or off the display 122 and the touch sensor 126. The microprocessor112 can thus turn on both the display 122 and the touch sensor 126, turnoff both the display 122 and the touch sensor 126, turn on the display122 while turning off the touch sensor 126, or turn off the display 122while turning on the touch sensor 126. The turning on and off of one ormore of the display 122 and the touch sensor 126 can include sending acommand to the one or more of the display 122 and touch sensor 126 tocause the respective element to turn off. Alternatively, the turning onand off of one or more of the display 122 and the touch sensor 126 caninclude removing power from the one or more of the display 122 and thetouch sensor 126, for example by opening or closing a switch providingpower to the display 122 or a switch providing power to the touch sensor126. In general, when the display driver 124 is turned on and offconcurrently with the display 122, while the sense controller circuit128 is turned on and off concurrently with the touch sensor 126.

The microprocessor 112 serves as a programmable controller for themobile device 10, in that it controls all operations of the mobiledevice 10 in accord with programming that it executes, for all normaloperations, and for operations involved in activating and operating ineach of the standard and VI modes of operation under consideration here.In the example, the mobile device 10 includes flash type program memory114, for storage of various “software” or “firmware” program routinesand mobile configuration settings. The mobile device 10 may also includea non-volatile random access memory (RAM) 116 for a working dataprocessing memory. Of course, other storage devices or configurationsmay be added to or substituted for those in the example. In a presentimplementation, the flash type program memory 114 stores firmware suchas a boot routine, device driver software, an operating system, and anyof a wide variety of other applications, such as client browser softwareand short message service software. The memories 114, 116 also storevarious data, such as server addresses, downloaded data such asmultimedia content, and various data input by the user. Programmingstored in the flash type program memory 114, sometimes referred to as“firmware,” is loaded into and executed by the microprocessor 112.

As outlined above, the mobile device 10 includes a processor, andprogramming stored in the flash memory 114 configures the processor sothat the mobile station is capable of performing various desiredfunctions, including in this case the functions involved in activatingand operating in each of the standard and VI modes of operation.

Unless otherwise stated, all measurements, values, ratings, positions,magnitudes, sizes, and other specifications that are set forth in thisspecification, including in the claims that follow, are approximate, notexact. They are intended to have a reasonable range that is consistentwith the functions to which they relate and with what is customary inthe art to which they pertain.

The scope of protection is limited solely by the claims that now follow.That scope is intended and should be interpreted to be as broad as isconsistent with the ordinary meaning of the language that is used in theclaims when interpreted in light of this specification and theprosecution history that follows and to encompass all structural andfunctional equivalents. Notwithstanding, none of the claims are intendedto embrace subject matter that fails to satisfy the requirement ofSections 101, 102, or 103 of the Patent Act, nor should they beinterpreted in such a way. Any unintended embracement of such subjectmatter is hereby disclaimed.

Except as stated immediately above, nothing that has been stated orillustrated is intended or should be interpreted to cause a dedicationof any component, step, feature, object, benefit, advantage, orequivalent to the public, regardless of whether it is or is not recitedin the claims.

It will be understood that the terms and expressions used herein havethe ordinary meaning as is accorded to such terms and expressions withrespect to their corresponding respective areas of inquiry and studyexcept where specific meanings have otherwise been set forth herein.Relational terms such as first and second and the like may be usedsolely to distinguish one entity or action from another withoutnecessarily requiring or implying any actual such relationship or orderbetween such entities or actions. The terms “comprises,” “comprising,”or any other variation thereof, are intended to cover a non-exclusiveinclusion, such that a process, method, article, or apparatus thatcomprises a list of elements does not include only those elements butmay include other elements not expressly listed or inherent to suchprocess, method, article, or apparatus. An element proceeded by “a” or“an” does not, without further constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises the element.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

While the foregoing has described what are considered to be the bestmode and/or other examples, it is understood that various modificationsmay be made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that the teachings may beapplied in numerous applications, only some of which have been describedherein. It is intended by the following claims to claim any and allapplications, modifications and variations that fall within the truescope of the present teachings.

What is claimed is:
 1. A method comprising: providing two modes ofoperation for a mobile device including a standard mode and a visuallyimpaired (VI) mode, wherein the VI mode is a mode of operation forvisually impaired users of the mobile device; upon the mobile devicepowering up from a low power state, automatically determining a mode ofoperation for the mobile device among the standard mode and the VI mode;and operating the mobile device in the automatically determined mode ofoperation, wherein operating the mobile device in the VI mode includesoperating the mobile device with a display screen of the mobile deviceturned off by default.
 2. The method of claim 1, wherein operating themobile device in the standard mode includes turning on the displayscreen of the mobile device in response to a set of conditions, andoperating the mobile device in the VI mode includes leaving the displayscreen of the mobile device turned off in response to the set ofconditions.
 3. The method of claim 2, wherein the display screen of themobile device and a touch-screen input interface of the mobile deviceare turned on in response to the set of conditions during operation ofthe mobile device in the standard mode, and the display screen remainsturned off while the touch-screen input interface is turned on inresponse to the set of conditions during operation of the mobile devicein the VI mode.
 4. The method of claim 1, wherein the automaticallydetermining a mode of operation of the mobile device comprisesdetermining a mode in which the mobile device operated immediately priorto entering the low power state.
 5. The method of claim 1, wherein upondetermining the mode of operation of the mobile device to be the VImode, performing steps of: temporarily turning on the display screen ofthe mobile device, and displaying a user notification on the displayscreen for a pre-determined period of time; and turning off the displayscreen upon determining that no user command is received to activate thestandard mode within the pre-determined period of time.
 6. The method ofclaim 1, wherein upon determining the mode of operation of the mobiledevice to be the VI mode, performing steps of: turning on the displayscreen of the mobile device, and displaying a user notification on thedisplay screen for a pre-determined period of time; and operating thedevice in the standard mode in response to receiving a user command toactivate the standard mode within the pre-determined period of time. 7.The method of claim 1, further comprising: during operation of themobile device in the VI mode, receiving a user command to temporarilyturn on the display screen; in response to receiving the user command,turning on the display screen of the mobile device and displayingcontent to the user on the display screen.
 8. The method of claim 7,wherein the receiving the user command to temporarily turn on thedisplay screen comprises detecting a pre-determined multi-finger gestureon a touch-sensitive input interface of the mobile device.
 9. A methodcomprising: providing two modes of operation for a mobile deviceincluding a standard mode and a visually impaired (VI) mode, wherein adisplay screen of the mobile device is turned off by default duringoperation in the VI mode; operating the mobile device in a user-selectedmode of operation, wherein operating the mobile device in theuser-selected VI mode includes: monitoring, using a forward facingcamera of the mobile device, whether a gaze of a user of the mobiledevice is directed towards the display screen of the mobile device; andupon detecting that the gaze is directed towards the display screen fora predetermined amount of time, initiating a change in mode of operationof the mobile device to the standard mode.
 10. The method of claim 9,wherein the initiating the change in mode of operation of the mobiledevice comprises: determining whether an application that is currentlyactive on the mobile device is included in a pre-determined list ofapplications for which a change in mode of operation can beautomatically initiated; and turning on the display screen of the mobiledevice in response to detecting that the gaze is directed towards thedisplay screen and that the application that is currently active on themobile device is included in the pre-determined list.
 11. The method ofclaim 9, wherein the initiating the change in mode of operationcomprises: temporarily turning on the display screen of the mobiledevice in response to detecting that the gaze is directed towards thedisplay screen, and displaying a user notification on the display screenfor a pre-determined period of time; and cancelling the change in modeof operation of the mobile device and turning off the display screenupon determining that the user does not wish to activate the standardmode.
 12. A mobile device comprising: a display screen operative todisplay information to a user of the mobile device; a memory; and aprocessor operative to execute machine-readable instructions stored inthe memory to cause the mobile device to perform functions includingfunctions to: provide two modes of operation for the mobile deviceincluding a standard mode and a visually impaired (VI) mode, wherein theVI mode is a mode of operation for visually impaired users of the mobiledevice; upon the mobile device powering up from a low power state,automatically determining a mode of operation for the mobile deviceamong the standard mode and the VI mode; and operating the mobile devicein the automatically determined mode of operation, wherein operating themobile device in the VI mode includes operating the mobile device withthe display screen of the mobile device turned off by default.
 13. Themobile device of claim 12, wherein operating the mobile device in thestandard mode includes turning on the display screen of the mobiledevice in response to a set of conditions, and operating the mobiledevice in the VI mode includes leaving the display screen of the mobiledevice turned off in response to the set of conditions.
 14. The mobiledevice of claim 13, further comprising: a touch-screen user interfaceoperative to receive touch input from the user of the mobile device,wherein the display screen of the mobile device and the touch-screeninput interface of the mobile device are turned on in response to theset of conditions during operation of the mobile device in the standardmode, and the display screen remains turned off while the touch-screeninput interface is turned on in response to the set of conditions duringoperation of the mobile device in the VI mode.
 15. The mobile device ofclaim 12, wherein the automatically determining a mode of operation ofthe mobile device comprises determining a mode in which the mobiledevice operated immediately prior to entering the low power state. 16.The mobile device of claim 12, wherein upon determining the mode ofoperation of the mobile device to be the VI mode, the processor causesthe mobile device to perform functions to: temporarily turn on thedisplay screen of the mobile device, and display a user notification onthe display screen for a pre-determined period of time; and turn off thedisplay screen upon determining that no user command is received toactivate the standard mode within the pre-determined period of time. 17.The mobile device of claim 12, wherein upon determining the mode ofoperation of the mobile device to be the VI mode, the processor causesthe mobile device to perform functions to: turn on the display screen ofthe mobile device, and display a user notification on the display screenfor a pre-determined period of time; and operate the device in thestandard mode in response to receiving a user command to activate thestandard mode within the pre-determined period of time.
 18. The mobiledevice of claim 12, wherein the processor causes the mobile device toperform further functions to: during operation of the mobile device inthe VI mode, receive a user command to temporarily turn on the displayscreen; in response to receiving the user command, turn on the displayscreen of the mobile device and display content to the user on thedisplay screen.
 19. The mobile device of claim 18, further comprising: atouch-screen user interface operative to receive touch input from theuser of the mobile device, wherein the receiving the user command totemporarily turn on the display screen comprises detecting apre-determined multi-finger gesture on a touch-sensitive input interfaceof the mobile device.